Ionian Islands
Data Efficacy for Language Model Training
Dai, Yalun, Huang, Yangyu, Zhang, Xin, Wu, Wenshan, Li, Chong, Lu, Wenhui, Cao, Shijie, Dong, Li, Li, Scarlett
Data is fundamental to the training of language models (LM). Recent research has been dedicated to data efficiency, which aims to maximize performance by selecting a minimal or optimal subset of training data. Techniques such as data filtering, sampling, and selection play a crucial role in this area. To complement it, we define Data Efficacy, which focuses on maximizing performance by optimizing the organization of training data and remains relatively underexplored. This work introduces a general paradigm, DELT, for considering data efficacy in LM training, which highlights the significance of training data organization. DELT comprises three components: Data Scoring, Data Selection, and Data Ordering. Among these components, we design Learnability-Quality Scoring (LQS), as a new instance of Data Scoring, which considers both the learnability and quality of each data sample from the gradient consistency perspective. We also devise Folding Ordering (FO), as a novel instance of Data Ordering, which addresses issues such as model forgetting and data distribution bias. Comprehensive experiments validate the data efficacy in LM training, which demonstrates the following: Firstly, various instances of the proposed DELT enhance LM performance to varying degrees without increasing the data scale and model size. Secondly, among these instances, the combination of our proposed LQS for data scoring and Folding for data ordering achieves the most significant improvement. Lastly, data efficacy can be achieved together with data efficiency by applying data selection. Therefore, we believe that data efficacy is a promising foundational area in LM training.
Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Video
Yao, David Yifan, Zhai, Albert J., Wang, Shenlong
This paper presents a unified approach to understanding dynamic scenes from casual videos. Large pretrained vision foundation models, such as vision-language, video depth prediction, motion tracking, and segmentation models, offer promising capabilities. However, training a single model for comprehensive 4D understanding remains challenging. We introduce Uni4D, a multi-stage optimization framework that harnesses multiple pretrained models to advance dynamic 3D modeling, including static/dynamic reconstruction, camera pose estimation, and dense 3D motion tracking. Our results show state-of-the-art performance in dynamic 4D modeling with superior visual quality. Notably, Uni4D requires no retraining or fine-tuning, highlighting the effectiveness of repurposing visual foundation models for 4D understanding.
AQUA-SLAM: Tightly-Coupled Underwater Acoustic-Visual-Inertial SLAM with Sensor Calibration
Xu, Shida, Zhang, Kaicheng, Wang, Sen
Abstract--Underwater environments pose significant challenges for visual Simultaneous Localization and Mapping (SLAM) systems due to limited visibility, inadequate illumination, and sporadic loss of structural features in images. Addressing these challenges, this paper introduces a novel, tightly-coupled Acoustic-Visual-Inertial SLAM approach, termed AQUA-SLAM, to fuse a Doppler Velocity Log (DVL), a stereo camera, and an Inertial Measurement Unit (IMU) within a graph optimization framework. The proposed system will be made open-source for the community. These vehicles are indispensable occasionally outside the camera's field of view leading to for tasks such as seabed mapping, pipeline and intermittent loss of visual tracking. Therefore, although visual cable inspections, biological and environmental monitoring, SLAM techniques have recently made tremendous progress and the maintenance of underwater infrastructure. A key in terrestrial settings [1], [2], [3], their performance and application area is the detailed visual inspection of subsea robustness are inevitably compromised in underwater due to structures, including offshore wind turbine foundations, where the complex and dynamic nature of aquatic environments. Considering cameras are widely equipped on underwater (IMU), known as visual-inertial SLAM (VI-SLAM) [4], [5], robots, visual Simultaneous Localization and Mapping can alleviate some of the challenges arising from transient, (SLAM) techniques emerge as natural solutions. The rapid attenuation of underwater SLAM systems, particularly against shortterm of light energy in water severely limits the visibility of visual disruptions, can be substantially enhanced [6]. However, most of the challenges for underwater vision, such Moreover, underwater vision often suffers from poor lighting as the limited visibility and the "marine snow", are longterm and blizzards of "marine snow" caused by small particles of effects that last at least from tens of seconds to a few organic matter in water, severely reducing image quality with minutes before being mitigated. VI-SLAM also encounters increased motion blur and dynamic image regions.
A Transformer-Based Framework for Greek Sign Language Production using Extended Skeletal Motion Representations
Pratikaki, Chrysa, Filntisis, Panagiotis, Katsamanis, Athanasios, Roussos, Anastasios, Maragos, Petros
Building on To address communication barriers between the DHH (Deaf and insights from previous research, we propose a deep learning model Hard-of-Hearing) and the hearing communities, the field of Sign for Sign Language Production (SLP), which to our knowledge is Language Processing has emerged at the intersection of linguistics, the first attempt on Greek SLP. We tackle this task by utilizing a computer vision, and machine learning. Sign Language Processing transformer-based architecture that enables the translation from encompasses a variety of tasks aimed at bridging the gap between text input to human pose keypoints, and the opposite. We evaluate DHH and hearing communities by enabling the automatic translation, the effectiveness of the proposed pipeline on the Greek SL dataset and generation of sign language. The most critical components Elementary23, through a series of comparative analyses and ablation of an effective sign language system are Sign Language Translation studies. Our pipeline's components, which include data-driven (SLT), and Sign Language Production (SLP). In this paper, we gloss generation, training through video to text translation and a primarily focus on Sign Language Production (SLP).